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Abstract 
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1 Introduction 

We consider a linear equation 

An = /, (1) 

where A : W n — > R m , and assume that equation |T]) has a solution, possibly non- 
unique. According to Hadamard [9l p. 9], problem ([1]) is called well-posed if the 
operator A is injective, surjective, and A^ 1 is continuous. Problem ([1]) is called 
ill-posed if it is not well-posed. Ill-conditioned linear algebraic systems arise as 
discretizations of ill-posed problems, such as Fredholm integral equations of the 
first kind, 

b 

k(x, t)u(t)dt = f(x), c<x<d, (2) 

where k(x, t) is a smooth kernel. Therefore, it is of interest to develop a method 
for solving ill-conditioned linear algebraic systems stably. In this paper we give a 
method for solving linear algebraic systems (Q]) with an ill-conditioned-matrix A. 
The matrix A is called ill-conditioned if n(A) >> 1, where k(A) := | \A\ | \A~ 1 \ | is 
the condition number of A. If the null-space of A, M{A) := {u : Au = 0}, is non- 
trivial, then k(A) = oo. Let A — ITEV* be the singular value decomposition 
(SVD) of A, UU* =U*U = 7, VV* = V*V = I, and £ = diag(ai, cr 2 , • . • , 
where o\ > 02 > • ■ ■ > cr m > are the singular values of A. Applying this SVD 
to the matrix A in (Q]), one gets 

/ = ^ PjUj and y = — v h (3) 

where /3j = (ui,f). Here (•,•) denotes the inner product of two vectors. The 
terms with small singular values <7j in ((3]) cause instability of the solution, be- 
cause the coefficients j3i are known with errors. This difficulty is essential when 
one deals with an ill-conditioned matrix A. Therefore a regularization is needed 
for solving ill-conditioned linear algebraic system |T]). There are many methods 
to solve (QJ stably: variational regularization, quasisolutions, iterative regular- 
ization (see e.g, [2], g], [B], [S])- The method proposed in this paper is based 
on the Dynamical Systems Method (DSM) developed in p.76]. The DSM for 
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solving equation |T]) with, possibly, nonlinear operator A consists of solving the 
Cauchy problem 

du 

u(t) = *(t,u(f)), u(0) = «o; «(t) := — , (4) 

where uq G H is an arbitrary element of a Hilbert space H, and $ is some 
nonlinearity, chosen so that the following three conditions hold: a) there exists 
a unique solution u(t) Vt > 0, b) there exists u(oo), and c) Au(po) = /. 

In this paper we choose $>(t,u(t)) = (A* A + a(i)/) _1 / — u(t) and consider 
the following Cauchy problem: 

u a (t) = -u a (t) + [A*A + a(t)I m }- 1 A*f, u a {0) = u o , (5) 

where 

a(t) > 0, and a(t) \ as t — > oo, (6) 

A* is the adjoint matrix and /„ is an to x to identity matrix. The initial element 
uq in (JSj) can be chosen arbitrarily in iVf^)- 1 , where 

N(A) := {u\Au = 0}. (7) 

For example, one may take uq = in ([5]) and then the unique solution to 
with u(0) = has the form 

u(t) = j'e-^T-^fds, (8) 

where T := T a := T + al, I is the identity operator. In the case of noisy 

data we replace the exact data / with the noisy data fs in ([5]), i.e., 

u s (t)= j\-^-'^ a) A*f s ds, (9) 

where t$ is the stopping time which will be discussed later. There are many 
ways to solve the Cauchy problem (JSJ). For example, one may apply a family of 
Runge-Kutta methods for solving ([5]). Numerically, the Runge-Kutta methods 
require an appropriate stepsize to get an accurate and stable solution. Usually 
the stepsizes have to be chosen sufficiently small to get such a solution. The 
number of steps will increase when tg, the stopping time, increases, see [2]. 
Therefore the computation time will increase significantly. Since lim^o ts = oo, 
as was proved in [9] , the family of the Runge-Kutta method may be less efficient 
for solving the Cauchy problem (|5|) than the method, proposed in this paper. 
We give a simple iterative scheme, based on the DSM, which produces stable 
solution to equation (fT]). The novel points in our paper are iterative schemes 
(JT3J) and (fTB")) (see below), which arc constructed on the basis of formulas © 
and ([9]), and a modification of the iterative scheme given in [8]. Our stopping 
rule for the iterative scheme (fl~3|) is given in ([85]) (see below). In [9l p. 76] the 
function a(t) is assumed to be a slowly decaying monotone function. In this 
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paper instead of using the slowly decaying continuous function a(t) we use the 
following piecewise-constant function: 

n-1 

a (n) (t) = ^a qj +1 X{t ., t . +l] (t), q e (0, 1), tj = -j ln(q), n e N, (10) 
where N is the set of positive integer, to = 0, ao > 0, and 

Xfe.^+dW = { £ othirwise! 11 ' (H) 

The parameter ao in (fTQ)) is chosen so that assumption (fT"7|) (see below) holds. 
This assumption plays an important role in the proposed iterative scheme. Def- 
inition (fTUjl allows one to write ([5]) in the form 

u n+1 = qu n + (1 - q)T- o \ n+1 A*f, u = 0. (12) 

A detailed derivation of the iterative scheme (|12p is given in Section 2. When 
the data / are contaminated by some noise, we use fg in place of / in ([5]), and 
get the iterative scheme 

ui +1 =qui + (l-q)T-^ +1 A*f s , 4 = 0. (13) 

We always assume that 

||/«-/||<*, (14) 

where f$ are the noisy data, which are known, while / is unknown, and 5 is 
the level of noise. Here and throughout this paper the notation ||z|| denotes the 
Z 2 -norm of the vector z £ M. m . In this paper a discrepancy type principle (DP) 
is proposed to choose the stopping index of iteration (fT3")) . This DP is based on 
discrepancy principle for the DSM developed in [llj . where the stopping time 
ts is obtained by solving the following nonlinear equation 

J** e-^-^a(s)\\Q^ s) fs\\ds = CS, C e (1, 2]. (15) 

It is a non-trivial task to obtain the stopping time t$ satisfying (|15[) . In this 
paper we propose a discrepancy type principle based on (|15p which can be easily 
implemented numerically: iterative scheme (|13p is stopped at the first integer 
ns satisfying the inequalities: 

ng— l 

(Q^- 1 - q^hoq^WQ-^JsWds < CS" 

(16) 

n—l x ' 

< E^" 1 - 1 n - j )a q j+1 \\Q-^ +1 fsl 1 < n < n 5 , 

3=0 
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and it is assumed that 

{l-q)a a q\\Q-l q f s \\>C8 E , C > 1, 6 6(0,1), a > 0. (17) 

We prove in Section 2 that using discrepancy- type principle (fTB|) , one gets the 
convergence: 

lini l|K - y\\ = 0, (18) 

o — >u 

where u 5 n is defined in (fl3|) . About other versions of discrepancy principles for 
DSM we refer the reader to [6], [12]. In this paper we assume that A is bounded. 
If the operator A is unbounded then fs may not belong to the domain of A* . In 
this case the expression A* fs is not defined. In [7J, [5] and [TU] solving (fTJ) with 
unbounded operators is discussed. In these papers the unbounded operator A is 
assumed to be linear, closed, densely defined operator in a Hilbert space. Under 
these assumptions one may use the operator A*(AA* +al)~ 1 in place of T~ X A*. 
This operator is defined for any / in the Hilbert space. 

In [5] an iterative scheme with a constant regularization parameter is given: 

ui +1 = aT- 1 u 5 n + T- 1 A*f s , (19) 

but the stopping rule, which produces a stable solution of equation {T]) by this 
iterative scheme, has not been discussed in [8 . In this paper the constant reg- 
ularization parameter a in iterative scheme (|19[) is replaced with the geometric 
series {aoq 71 }^^, a > 0, q G (0, 1), i.e. 

u s n+1 = a oq n T-^ qn u s n + T-\n A* f s . (20) 

Stopping rule (|85[) (see below) is used for this iterative scheme. Without loss 
of generality we use ao = 1 in (|20p . The convergence analysis of this iterative 
scheme is presented in Section 3. In Section 4 some numerical experiments are 
given to illustrate the efficiency of the proposed methods. 



2 Derivation of the proposed method 

In this section we give a detailed derivation of iterative schemes (fT2"j) and (fT5|) . 
Let us denote by y € M. m the unique minimal- norm solution of equation (|TJ) . 
Throughout this paper we denote T a t t \ := A* A+a(t)I m , where I m is the identity 
operator in K m , and a(t) is given in (fTU]) . 

Lemma 2.1. Let g(x) be a continuous function on (0, oo), c > and q € (0, 1) 
be constants. If 

lfag(x)=g(0):=g o , (21) 

then 

n-l 

Urn J2 ~ <T~ j ) 9^ 3+1 ) = .90- (22) 
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Proof. Let 
and 

Then 



Wj -(n) :=q n -i -q n+1 ~i, Uj (n) > 0, 



(23) 
(24) 



|F n+1 (n)- 5o | < |F,(n)| + 



^^•(71)5(0^') - .go 
3=1 



Take e > arbitrary small. For sufficiently large 1(e) one can choose n(e), such 
that 

\F m (n)\ ^ f> Vn>n(e), 

because lim n _>oo q" = 0. Fix Z = /(e) such that \g{cq*) — go | < § for j > /(e). 
This is possible because of (l2"Tj) . One has 

< f> n>n(e) 

and 



XI Uj(n)g(cq j ) - g 



< Ujin^gicq 3 ) - g \ + | ^ ^(n) - l|| 5o | 

n 

< I + l5ok n -' (e) < e, 



if n is sufficiently large. Here we have used the relation 

n 

£ Wi (n)=l-? B + 1 - 1 . 



Since e > is arbitrarily small, Lemma l2.1l is proved. 
Let us define 



□ 



t*"-)^, jA'/ib, t n = -nln(g), ?g(0,1). (25) 



G 



Note that 



u„ 







tn-l rt 



e -(t»-tn-i) / e - < - t "- 1 - s) T- ( \A*fds+ e-^-^T-*, ,A*fds 

Jo a(S) Jtn-l 



e -(*»-*»-i) Un _ 1 + / e -(*»-)T^) w A7ds 



Using definition (fT0|) . one gets 

u» = e-^-*"- 1 )^! + [1 - e-^-^]T-^A*f 

n n 

= -^—run-! + (1 - — r )T- 1 n nA*f. 

qn—l g7l— 1 ' "09 J 

Therefore, can be rewritten as iterative scheme (|T2|) . 
Lemma 2.2. Let u n be defined in (|12[) and Ay = /. Then 

n-l 

IK - V|| < <T\\v\\ + E (? n "^ 1 - "o^'+'llT-^j/H, Vn > 1, (26) 
and 

|| Un ~ y\\ ^0 as 7i — ► oo . (27) 
Proof. By definitions (|23|) and (fTU)) we obtain 

w » - jf e-^-^ijAV* = E (^T ~ 5) T ^+^ (28) 
From ([28]) and the equation Ay — f, one gets: 



j'=o 



3 =o ^ 1 J 

71—1 71—1 

- a a q- 



3=0 j=0 
n-l 

3=0 

Thus, estimate (|2^|) follows. To prove (|2"T|) , we apply Lemma I2TT1 with g(a) 
a ll^ 1 (T 1 yll- Since y _L Af(A), it follows from the spectral theorem that 

-,2 

^2/ 



lima («) = lim / - — —=d(E s y,y) = 11^(^)2/11 = 0, 

a— >0 a^O Jq (a + S) z 
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where E s is the resolution of the identity corresponding to A* A, and P is the 
orthogonal projector onto N(A). Thus, by Lemma [27X1 ([27)) follows. □ 



Let us discuss iterative scheme (fT3|) . The following lemma gives the estimate 
of the difference of the solutions u s n and u n . 

Lemma 2.3. Let u n and u s n be defined in (112p and ()13|) . respectively. Then 



|un~«n|| < x ^ m w ^ n>Q, (29) 



where w n := (1 — q) ., . 

Proof. Let _ff„ := — u n || . Then from the definitions of u s n and u n we get the 
estimate 

H n+ i < q\\u 5 n - u n \\ + (1 - q)\\T-^ n+1 A*(f s -f)\\< qH n + w n . (30) 

Let us prove inequality ()29|1 by induction. For n = one has uo = = 0, so 
(ED) holds. For n = 1 one has \\uf -ui\\ < (1 - £?)— r — , so ([Ml) holds. If ® 



2y a g : 

holds for n < k, then for n = fc + 1 one has 



/ g 3 / 2 \ 1 
tffc+i < qH k + w k < y i q3/2 + lj ™fc = ^3/2 ^ 

1 Wk 1 /- 



(31) 



1 — g 3 / 2 tWfe+x 1 — <Z 3 / 2 

Hence (|29l) is proved for n > 0. □ 



2.1 Stopping criterion 

In this section we give a stopping rule for iterative scheme given in ()13[) . Let 
Q := AA*, Q a -=Q + al m , and 

Gn := j\-^a(s)\\Q-l s) f s \\ds 

n-l (32) 

= E^" 1 - ^~ j ) a oq j+1 \\Q^ qj+1 fs\\, n > 1, 
i=o 

where i„ = — nlng, g € (0,1) and «o > 0. Then stopping rule p6[) can be 
rewritten as 

G„ a < C<5 £ < G„, 1 < n < n s , e e (0, 1), C* > 1, G x > C*<S £ . (33) 
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Note that 

n 

G n+1 = J2(l n - j q n+l - j )uoq j+1 \\Q-^ + Js\\ 



n-l 

= £(g»-' q^-^ao^WQ-^JsW + (1 - q)a q n+1 \\Q- 1 qn+1 f S \\ 

3=0 

= qG n + (l-q)a q n + 1 \\Q- 1 oqn+1 fsl 

SO 

G n =qG n - 1 + (l-q)a q n \\Q-i qn f 5 \\, n > 1, G = 0. (34) 
Lemma 2.4. Let G n be defined in (13"4")) . Then 

G n < ^-^^q^M+5, n>l, ge(0,l). (35) 

Proof. Using the identity 

-aQ- 1 = AT' 1 A* - I m , a > 0, T := AM, T„ := T + al m , 
the estimates 

a\\Q?\\<l, \\fs-f\\<S, 

and 

allAT- 1 !! < ^, 
:= Q + a/ TO , we get 

G, 



= yL4*, 


Qa 




-i- 


-(1 


qG n - 


-i- 


-(1 


qG n - 


-l- 


-(1 


qG n - 


-l- 


-(1 


qG n - 


-i- 


-(1 


qG n - 


-i- 


-(1 


qG n - 


-i- 


-(1 


qG n - 


-l- 


H(l 


qG n - 


-l- 


-(1 


qG n - 


-i- 


-(1 


qG n - 


-l- 


-(1 


qG n - 


-i- 


-(1 


qG n - 


-i- 


-(1 



aoq"- J J 1 1 

' -Ml 

+ (1 - q)\\(AA* + a q n I - a q n I)Q- n L fs - Ml 



-a q" 
?aoq™.MI 

?aoV(/5-/ + /)H 

+ (1 - 5)a 9 n ||QaoV(/5 - /)H + (! - ?)«o«"||Q« V/ll 
+ (l- (? )5+(l- (Z )|| J 4T Q - 1 9 „^/-/ll 

T-^AMy-y)!! 

-aoffT-^y)!! 



(36) 



Hi-^ + V^Hbl 
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Therefore, 

G n - 5 < 9 (G n _i -S) + Vq^f^\\y\\, n>l, G = 0. (37) 
Let us prove relation (|35() by induction. From relation (I3T[) we get 

Gi-(5 < - q S + y——\\y\\ < -qS+- -■*——\\y\\ < F -*-r— y . (38) 

Thus, for n = 1 relation ([33)1 holds. Suppose that 



Then by inequalities (|3~7| and (f3T))) we obtain 



A' 



1 \/ a q k _V"og M „ 



(40) 



y\\ = i F „ / , . fVQog + 



2 1- Vq2y/a q k+ 



1 



< 



1 V«o9 fe+1 



IIJ/II 



Thus, relation (|35|) is proved. □ 

Lemma 2.5. Lei G n be defined in (|34|) . g G (0, 1), and ceo > oe chosen such 

that Gi > C5 e , e G (0, 1), C > 1. TTien i/iere exists a unique integer n c such 
that 

Gn a -i < G Uc and G„ c > G„ c+ i, rt c > 1. (41) 

Moreover, 

G n+ i < G n , Vn > n c . (42) 



Proof. From Lemma 12.41 we have 

G n < — !— VE^M + s, n >l, ge(0,l). 
1 - a/^ 2 

Since Gi > G£ e and limsup„^ oc G n < S < C6 £ , it follows that there exists 
an integer n c > 1 such that G„ c _i < G„ c and G„ c > G„ c +i. Let us prove the 
monotonicity of G„, for n > n c . We have G nc +i — G„ c < 0. Using definition 
(|34|) . we get 



G„ c+1 - G„ c - qG nc + (1 - g)ao<z" c+1 ||Q-Vc +1 MI " G„ c 

(43) 



= (1 - q) (aoq n ° +X \\Q- l oqnc+1 f 5 \\ ~ G„ c ) < 0. 
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This implies 



a q 



n c + l I 



aoQ'l+JsW ~ G nc <0. (44) 



Note that 

G n+1 - G n = (1 - q){a oq n+1 \\Q-l qn+1 fs\\ - G n ). 

Therefore, to prove the monotonicity of G n for n > n c , one needs to prove the 
inequality 

avq n+1 \\Q- a l qn+ Js\\-G n <0, Vn>n c . 
This inequality is a consequence of the following lemma: 
Lemma 2.6. Let G n be defined in ([M]) , and (|44[) holds. Then 

*oq n+1 \\Q^ qn+ j8\\-G n <0, Vn>n c . (45) 

Proof. Let us prove Lemma 12.61 by induction. Let 

A l :=a 0(Z " +1 ||g; o 1 g „ +1 / 5 ||-G„ 

and 

%) ^^iiq- 1 /,!! 2 . 

The function /i(a) is a monotonically growing function of a, a > 0. Indeed, by 
the spectral theorem, we get 



h(ax) = ai\\Q- a lh\\ 2 = / Ql d(F fl /f, / 4 ) 
Jo (Oi + s) 2 



(46) 



where F s is the resolution of the identity corresponding to Q := AA*, because 

a 2 a 2 

( ai 4 L s )2 < (a 2 +s) 2 if < ai < a2 and s > 0. By the assumption we have 
D nc = aoq n " +1 \\Q~^ qnc+1 fs\\ - G 7lc < 0. Thus, relation (gSj) holds for n = n c . 
For n = n c + 1 we get 

A, c+1 = a ^ +2 ||Q- V c+3 MI - (1 - qhoq n < +1 \\Q-^ +1 fs\\ - qG nc 



y/h(a q^+ 2 ) - y/h(a q n c+i) + qy / h(a q n <= + 1 ) - qG Tr 



= ^h(a q n "+ 2 ) - ^h(a q n ° +1 ) + q(^h(a q n " +1 ) - G„J (47) 

= y/h(a q n *+ 2 ) - y/hiaaqO'+^+qD^ 

= y/h(a q n °+ 2 ) - y/h(a q n " +1 ) + qD nc < 0. 

Here we have used the monotonicity of the function h(a). Thus, relation (|45[) 
holds for n = n c + 1. Suppose 

D n < 0, n c < n < n c + k — 1. 
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This, together with the monotonically growth of the function h (a) := a 2 ||<3 9 Vail 2 , 
yields 

D na+k = ao^' +k+l \\Q^ +h+1 fe\\ - G na+k 



/ h(a q^+ k + 1 ) - (1 - q)yjh(a q^+ k )~ g G„ c +fc-l 
= yj 'h{a q n c+ k + 1 ) - yj 'h(aoq n °+ k ) + q{yj ' h{a q n ^+ k ) - G„ c+fc „i) (48) 
/ h(a <t° +k+1 ) ~ ^h{a q n "+ k ) + qD nc+k ^ 
1 ' h(a q n c+ k + 1 ) - yj h{a q n "+ k ) + ?D no+ *_i < 0. 
Thus, D n < 0, n > 1. Lemma l2~i)l is proved. □ 
Let us continue with the proof of Lemma 12.51 From relation (|34[) we have 
G„+i - G n = (q- 1)G„ + (1 - g)aog" +1 ||g; o V +1 MI 
= (1-9) ("o?" +1 ||Q; o V +1 / 5 ||-G n ). 

Using assumption (|44[) and applying Lemma l2.6l one gets 

G„+i - G„ < 0, Vn > n c . 

Let us prove that the integer n c is unique. Suppose there exists another integer 
rid such that G n<J _ i < G n<J and G„ d > G n<j +i. One may assume without loss 
of generality that n c < rid- Since G„ > G n +i, Vn > n c , and n c < n<2, it follows 
that G n<J _i > G„ d . This contradicts the assumption G„ d _i < G n<J . Thus, the 
integer n c is unique. Lemma 12.51 is proved. □ 



Lemma 2.7. Let G„ 6e defined in (I34p . // ao is chosen such that relations 
Gi > G<5 e , G > 1, £ G (0, 1), /io/ds i/ien t/iere exists a unique ng satisfying 
inequality 



Proof. Let us show that there exists an integer n$ so that inequality (|33l) holds. 
Applying Lemma |2.4[ one gets 

limsupG„<(5. (49) 

n — >oo 

Since Gi > C6 e and limsup,^^ G„ < <5 < CS e , it follows that there exists an 
index n$ satisfying stopping rule (|33[) . The uniqueness of the index n$ follows 
from the monotonicity of G n , see Lemma 12.51 Thus, Lemma 12.71 is proved. □ 

Lemma 2.8. Let Ay = f, y _L Af(A), and ns be chosen by rule ()33j) . Then 

Km?"'=0, 9 6(0,1), (50) 

o — >0 



lim = co. (51) 

5^0 
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Proof. From rule (|33[) and relation (j3~i|) we have 

?C<J B + (1 - q)a Q q n *\\Q-i qn J s \\ < qG ns . x + (1 - q)a q n ° \\Q~^ S f s \\ 



(52) 



so 

(1 - q)a oq ^\\Q-^J s \\ < (1 - g)C<P. (53) 

Thus, 

ao9 n 'IIQaoW«H < CTE - ( 54 ) 
Note that if / ^ then there exists a Ao > such that 

FAo/t^O, (F Ao /,/) :=C>0, (55) 

where £ is a constant which does not depend on 5, and F s is the resolution of 
the identity corresponding to the operator Q := AA*. Let 

h(S,a) :=o?\\Q- l f s f. 

For a fixed number ci > we obtain 

^(<5, Cl )= c 2 ||Q Cl / 5 || 2 = / , * , 2 d(F s f s ,f s )> / —l—d(F,f B ,f e ) 

> , r°d(fi/ i ,/ i )= f ii yr , ^>o. 

(ci + A ) 2 7 (ci + A ) 2 

(56) 

Since -F\ is a continuous operator, and ||/ — fg\\ < S, it follows from ([55]) that 
lim(F Xo f 5 Js) = (F\ f, /) > 0. (57) 

o — >0 

Therefore, for the fixed number c\ > we get 

h(5, Cl )>c 2 >0 (58) 

for all sufficiently small 6 > 0, where C2 is a constant which does not depend on 
S. For example one may take c<x = | provided that (|55[) holds. Let us derive 
from estimate (p3f and the relation (j55|) that g" 5 — > as <5 — * 0. From (|5^|) we 
have 

< /i^aog™ 15 ) < (C<5 £ ) 2 . 

Therefore, 

lim h(S,a q ns ) = 0. (59) 

(5^0 

Suppose lim^o q ns ^ 0. Then there exists a subsequence Sj — > such that 

a q ni i > Cl > 0, (60) 
where ci is a constant. By (|58|) we get 

h(5j,ctoq ns j ) > c 2 > 0, (5j — > as j — > oo. (61) 
This contradicts relation (|59|) . Thus, lim^o 9™ i = 0. Lemma l2~8l is proved. □ 
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Lemma 2.9. Let n$ be chosen by rule (|33|) . Then 

S 



as S -► 0. (62) 



Proof. Relation ((35)1 , together with stopping rule (|33|) . implies 



1 



G<5 £ < G n5 _x < 1 f-^= V llvll + *• (63) 



Then 



This yields 



< TTT^i ifiL TT ' £6(0,1). (64) 



^^T- 2(1 -^(G-l) 



(5 <5 (5^ £ 

lim , = lim — = < lim — — — r — w = 0. (65) 

"v'arf 17 v/^^ 1 " 5-0 2^(1 -7g)(C-l)"' y " 

Lemma T2. 91 is proved. □ 

Theorem 2.10. Let y _L Af(A), \\f s - f\\ < S, \\f s \\ > CS E , C > 1, £ G (0, 1). 
Suppose ng is chosen by rule (|33p . TTien 

lim ||< -2/11=0, (66) 

where u s n is given in (|13[) . 

Proof. Using Lemma 12.21 and Lemma 12.31 we get the estimate 

ll< - v\\ < IK - ^ II + IK* - y\\ < T^f^ (1 - q) 2 q J^ + lKs ~ y|1 

:= ii+Ia, 

where Ji := - g)^-^== and 7 2 := ||wn, — 2/11- Applying Lemma 

one gets lim^o Ii = 0. Since ng — ► 00 as <5 — * 0, it follows from Lemma [2~2l that 
lima-^o L2 = 0. Thus, lima^o ||u* — y|| = 0. Theorem 12. 101 is proved. □ 

The algorithm based on the proposed method can be stated as follows: 

Step 1. Assume that dHJ) holds. Choose C G (1, 2) and e G (0.9, 1). Fix q G (0, 1), 
and choose otQ > so that (fTT|) holds. Set n = 1, and uq = 0. 

Step 2. Use iterative scheme (fTBf to calculate w n . 

Step 3. Calculate G„, where G„ is defined in ([34]) . 



(67) 



Step 4. If G„ < CS £ then stop the iteration, set ng — n, and take u 5 ns as the 
approximate solution. Otherwise set n = n + 1, and go to Step 1. 
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3 Iterative scheme 2 

In [8] the following iterative scheme for the exact data / is given: 

u n+1 = aT- x u n + T~ 1 A*f, ui = «i ± Af(A), (68) 

where a is a fixed positive constant. It is proved in [5] that iterative scheme 
(|S"5)) gives the relation 

lim |jw„-y|| =0, y ±M(A). 

n — >oo 

In the case of noisy data the exact data / in (|68[) is replaced with the noisy data 
fs, i-e. 

<+i = aT- 1 u s n + T- 1 A*f s , u 1 =u 1 ±tf(A), (69) 

where — /|| < S for sufficiently small S > 0. It is proved in [5] that there 
exist an integer rig such that 

lim ||< - j/ll = 0, (70) 

o — >u 

where is the approximate solution corresponds to the noisy data. But a 
method of choosing the integer ns has not been discussed. In this section we 
modify iterative scheme by replacing the constant parameter a in (|6"8|) with 
a geometric sequence {q n ~ 1 }'^ = i, i.e. 

u n+1 - jT^o,, + T-jA*f, ui = 0, (71) 

where q € (0, 1). The initial approximation u\ is chosen to be 0. In general one 
may choose an arbitrary initial approximation u\ in the set M{A)^. If the data 
are noisy then the exact data / in (|71|) is replaced with the noisy data fs, and 
iterative scheme (|69|) is replaced with: 

u s n+1 = q n T- n l u 5 n + T- n l A*f s , u{ = 0. (72) 

We prove convergence of the solution obtained by iterative scheme ([71]) in The- 
orem [23] for arbitrary q G (0, 1), i.e. 

lim |K -y|| -0, Vg 6(0,1). 

n — >oo 

In the case of noisy data we use discrepancy-type principle (|85[) to obtain the 
integer ns such that 

lim ||< -y||=0. (73) 

We prove relation ([T3"|) , for arbitrary g€ (0, 1), in Theorem 13.61 

Let us prove that the sequence u n , defined by iterative scheme (|71[) . converges 

to the minimal norm solution y of equation (fT]) . 

Theorem 3.1. Consider iterative scheme ([71]) . Lety±AT(A). Then 

lim \\u n -y\\=0. (74) 
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Proof. Consider the identity 

y = aT- 1 y + T- 1 A*f, Ay = f. (75) 

Let w n := u n - y and B n := q n T~,} . Then w n +i = B n w ni w\ = y — u\ = y. 
One uses (|75|) and gets 

||y-u n f = \\B n ^B n ^ 2 . . . B lWl \\ 2 = \\B n - 1 B n - 2 ...B 1 y\\ 2 



q n 1 q n 2 q 



X2/ ? „-2 X2 / ? \ 2 (7(il 



d(E s y,y) 

d(E s y,y) 



/•°° a 2 ' 1 

where E s is the resolution of the identity corresponding to the operator T :— 
A* A. Here we have used the identity (|75|) and the monotonicity of the function 
:= (x+s)' 1 ' s — ^- F rom estimate ([76^1 we derive relation ([71)) . Indeed, write 

_,2n ^& ^2n /-oo ^2n 



/„ ^Tsr d{Esy,y) = J ¥Tsr d{Esmy) + k I^F rf( ^' y) ' 

(77) 

where b is a sufficiently small number which will be chosen later. For any fixed 



b > one has — j— < < 1 if s > 6. Therefore, it follows that 



oo 9 2„ 



(9 



—^d(E s y, y) -> as n -> oo. 



(78) 



On the other hand one has 



6 2n z-6 

« d{E s y,y)< / dffi.y.y). (79) 

Since y _L A/"(A), one has lim£,_>o J b d(E s y, y) = 0. Therefore, given an arbitrary 
number e > one can choose b(e) such that 

6(e) 2n 

, H ,„ d(E s y,y) < -. (80) 
(<7 + s) 2 ™ V W ' y/ 2 V 7 

Using this 6(e), one chooses sufficiently large n(e) such that 

Since e > is arbitrary, Theorem 13. II is proved. □ 
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As we mentioned before if the exact data / are contaminated by some noise 
then iterative scheme (|72l) is used, where \\fs — f\\ < S. Note that 

S 5 
||t4+i - «n+l|| < q n \\T^(u S n - u n )\\ + — < ||< - u n \\ + — . (82) 

To prove the convergence of the solution obtained by iterative scheme (|72p . we 
need the following lemmas: 

Lemma 3.2. Let u n and u s n be defined in Q71[) and (|72p . respectively. Then 

K-n»||< 1 ^ / _ 2 j_ 7r , n>l. (83) 

Proof. Let us prove relation (|83[) by induction. For n = 1 one has — u i = 0- 
Thus, for n = 1 the relation holds. Suppose 

ll«f-m||< ^ ^ ro L i<l<k. (84) 

Then from (O and ((84l) we have 



1*4+1 - "fc+ill < 11*4 - "fell + T-Tt: ^ ^ 



<Vq-, 



5 



•(l-v^W^ 

Thus, 

Lemma l3~2l is proved. □ 



Let us formulate our stopping rule: the iteration in iterative scheme (|72[) is 
stopped at the first integer ns satisfying 

\\AT- n ]A*f s - Ml < C8 e < \\AT-jA\fs - f s \\, 1 < n< n s , C> 1, e € (0, 1), 

(85) 

and it is assumed that ||MI > CS e . 

Lemma 3.3. Let u 5 n be defined in (f?2|). and W n := \\AT qn 1 A*f s - Ml- Then 

Wn+l <W n , n> 1. (86) 

Proof. Note that 

W n = \\AA*Q-}f s - M| = ||(Q 9 „ - q n I m )Q-,}f s - M| = ||<f Q-JMI, (87) 
where Q := AA* , and Q a := Q + a/ m . Using the spectral theorem, one gets 

poo -2(71+1) /•<» 2n 
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where F s is the resolution of the identity corresponding to the operator Q := 
AA* . Here we have used the monotonicity of the function g{x) — i ^ » 3 , s > 
0. Thus, 

W n+1 <W n , n> 1. (89) 



Lemma 13.31 is proved. □ 

Lemma 3.4. Let u s n be defined in ([72j) , and \\f$\\ > CS e , e £ (0,1), C > 1. 
TTien i/iere exists a unique index ns such that inequality (I85[) holds. 



Proof. Let e n := AT^A*/* - /«. Then 

e„ = q n Qqnfs, (90) 
where Q Q := AA* + a/. Therefore, 

IM| < ||g n Q^(/*-/)|| + ||g n Q^/|| 

(91) 



< \\fs-f\\ + h n Q^Ay\\<5+^-\\yl 
where the estimate ||Q~ 1 A| = HAT^ 1 ! < was used. Thus, 

limsup ||e n || < S. 

n — >oo 

This shows that the integer ns, satisfying (|85p . exists. The uniqueness of ns 
follows from its definition. □ 

Lemma 3.5. Let v? n be defined in {72]). If ns is chosen by rule (|85|) then 

lim — = = 0. (92) 



Proof. From (|91|) we have 



V 



n-1 



AT-^A*/, - M| < 5 + || ei ||, (93) 



where e\ := u± — y = —y. It follows from stopping rule (|85[) and estimate ([93 
that 



C^ e < UAT-^A*/* - Ml < + ^. ( 94 ) 



Therefore, 



(C-l)*"< V5_|| ei ||, (95) 

and so 

7^^276^' (96) 
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This implies 

Vqq^ ~ 2<? 1 /2(c- l)5 s 2gVa(C-l) ' V ' 
Thus, - * as 5 — » 0. Lemma [531 is proved. □ 

The proof of convergence of the solution obtained by iterative scheme (|72[) 
is given in the following theorem: 

Theorem 3.6. Let u s n be defined in ([72]), y _L Af(A), \\f s \\ > CS £ , e € (0,1), 
C > 1, q (0,1). If n$ is chosen by rule (|85[) . f/ien 

||4 - 2/H -> as 5 - 0. (98) 

Proof. From Lemma 13.21 we get the following estimate: 

IK-V|| < lK-«n,ll + ll«»*-vll < 1 ^ 6 +\Wn s ~y\\ :=h+h, (99) 



where Ji '■= \^jq i^/q n & an< ^ ^ 2 := W Un s ~ v\\- ^ v Lemma [3~5l one gets I\ — > 
as (5 — ► 0. To prove lim^o I2 = one needs the relation lima^o n§ = 00. This 
relation is a consequence of the following lemma: 



Lemma 3.7. If ns is chosen by rule J85J), then 

q ns -► as 6 -> 0, (100) 



■SO 



lim ns = 00. (101) 

S-vO 



Proo/. Note that 

AT- 1 ^.^ - /, = AA*Q~ l fs -fs = (AA* + al m - a/^Q" 1 /* - / 4 
= /<5 - aQ~ l fs - /* = -aQ~ 1 f s , 

where a > 0, Q := and Q a := Q + al. From stopping rule ([85]) we have 
< IIAT-^A*/* - / a || < CS £ . Thus, 

lim \\AT- n \A*f s - M| = lim H^Q"^ M| = 0. (102) 



Using an argument given in the proof of Lemma 12.8] (see formulas (j54p ~(|6i p 
in which ctQ = 1), one gets lim^o q ns = 0, so lim^o n 8 = °°- Lemma 13.71 is 
proved. □ 



Lemma [377] and Theorem 13.11 imply I2 — > as 5 — » 0. Thus, Theorem 13.61 is 
proved. □ 
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4 Numerical experiments 



In all the experiments we measure the accuracy of the approximate solutions 
using the relative error: 

Rel-Err = ^- , 

\\y\\ 

where ||.|| is the Euclidean norm in M. n . The exact data are perturbed by some 
noises so that 

n/«-/n<<y, 

where 

fs = f + S^, 
INI 

S is the noise level, and e S K™ is the noise taken from the Gaussian distribution 
with mean and standard deviation 1. The MATLAB routine called "randn" 
with seed 15 is used to generate the vector e. The iterative schemes ([T3|) and (|72[) 
will be denoted by I Si and IS%, respectively. In the iterative scheme I Si, for 
fixed q £ (0, 1), one needs to choose a sufficiently large ao > so that inequality 
Q17[) hold, for example one may choose ao > 1. The number of iterations of 
I Si and IS2 are denoted by Iteri and Iteri, respectively. We compare the 
results obtained by the proposed methods with the results obtained by using the 
variational regularization method (VR). In VR we use the Newton method for 
solving the equation for regularization parameter. In [4] the nonlinear equation 

\\Au VR (a) - M| 2 = (CS) 2 , C = 1.01, (103) 

where uvr{o) := T~ 1 A*fg, is solved by the Newton's method. In this paper the 
initial value of the regularization parameter is taken to be ao = -^j, where kg is 
the first integer such that the Newton's method for solving (|103[) converges. We 
stop the iteration of the Newton's method at the first integer rig satisfying the 
inequality \\\AT~^A*fg - f s \\ 2 - {C5) 2 \ < 10- 3 (C<S) 2 , a Q := a . The number 
of iterations needed to complete a convergent Newton's method is denoted by 
Iter VR . 

4.1 Ill-conditioned linear algebraic systems 



Table 1: Condition number of some Hilbcrt matrices 



n 


k(A) = \\A\\\\A-i\\ 


10 


1.915 x 10 13 


20 


1.483 x 10 28 


70 


8.808 x 10 105 


100 


1.262 x 10 150 


200 


1.446 x 10 303 
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Consider the following system: 

H^u = /, (104) 

where 

is a Hilbert matrix of dimension m. The system (|104[) is an example of a severely 
ill-posed problem if m > 10, because the condition number of the Hilbert matrix 
is increasing exponentially as m grows, see Table ([T]). The minimal eigenvalues 
of Hilbert matrix of dimension m can be obtained using the following formula 

A min (ff (m) ) -2 15 / 4 7r 3 / 2 V^(\/2 + l)-( 4m + 4 )(l + (l)). (105) 
This formula is proved in |3]. Since ^H^) = ^[g^'j , it follows from ([105]) 

3.5255 m 

that the condition number grows as 0( e — ). The following exact solution is 
used to test the proposed methods: 

y e R m , where yu = \Thk, k — 1, 2, . . . , m. 

The Hilbert matrix of dimension m — 200 is used in the experiments. This 
matrix has condition number of order 10 303 , so it is a severely ill-conditioned 
matrix. In Table 2 one can see that the number of iterations of the iterative 



Table 2: Hilbert matrix problem: the number of iterations and the relative 
errors with respect to the parameter q (ao = 1, S — 10 -2 ). 



q I Si IS 2 





REl.Err 


Iteri 


REl.Err 


Iter 2- 


.5 


0.031 


24 


0.032 


23 


.25 


0.031 


13 


0.032 


13 


.125 


0.032 


9 


0.032 


9 



scheme ISi and IS2 increases as the value of q increases. The relative errors 
start to increase at q = .125. By these observations, we suggest to choose the 
parameter q in the interval (.125, .5). In Table 3 the results of the experiments 
with various values of S are presented. Here the parameter e was .99. The 
geometric sequence {.25 n ~ 1 }^ D =1 was used in the iterative schemes ISi and LS2. 
The parameter C in (|16p and (|85[) were 1.01. The parameter kg in the variational 
regularization method was 1. One can see that the relative errors of I Si and IS2 
are smaller than these for the VR. The relative error decreases as the noise level 
decreases which can be seen on the same table. This shows that the proposed 
method produces stable solutions. 
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Table 3: ICLAS with Hilbert matrix: the relative errors and the number of 
iterations 



6 


ISi 




IS 2 




VR 






REl.Err 


Iter i 


REl.Err 


Iter 2 


REl.Err 


ItervR- 


5% 


0.038 


11 


0.043 


11 


0.055 


13 


3% 


0.037 


12 


0.034 


12 


0.045 


14 


1% 


0.031 


13 


0.032 


13 


0.034 


15 



4.2 Predholm integral equations of the first kind (FIEFK) 

Here we consider two Fredholm integral equations : 

a) 

f(s) = J k(t- s)u(t)dt, (106) 



where 



/<<•-) H J + cos( ^' (107) 



and 



b) 



/(js) = < { (e + ^Cl-icosd^-^smCf), |*|£6; (10g) 



f(s)= I k(s,t)u(t)dt,s € (0,1), (109) 



o 



where 



and 



< t-l), ^>t (U0) 



f(s) = (s 3 - s)/6. (Ill) 



The problem a) is discussed in [5] where the solution to this problem is u(x) = 
k(x). The second problem is taken from [1] where the solution is u{x) = x. The 
Galerkin's method is used to discretized the integrals (|106l) and (|109p . For the 
basis functions we use the following orthonormal box functions 



and 



Hs)-=\ y^' p*- 1 * 3 *}' (H2) 

otherwise, 



^(t) : = J V ; (113) 

I 0, otherwise, 
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where Si = d\ + U = + i— , i = 0, 1, 2, . . . , to. In the problem a) the 
parameters ci, C2, <2i, c?2, d^ and d^ are set to 12, 6, —6, 12, —3 and 6, 
respectively. In the second problem we use d\ =(£3 = and c\ = c-i = di = 
d\ = 1. Here we approximate the solution u(t) by u = 52j=i c ji } j{t)- Therefore 
solving problem (|106|) is reduced to solving the linear algebraic system 

Ac = f, £,fe« m , (114) 

where in problem a) 

Aij = f f k(t- s)<j>i(s)ij}j(t)dsdt 

J -3 J -6 

and /j = J^ 6 f(s)<pi(s)ds, i,j = l,2,...,m, and in problem &) 

Aij = / / k(s,t)(pi(s)ipj(t)dsdt 
Jo Jo 

and /j = Jq 1 f(s)4>i(s)ds, and Cj = c,- i, j = 1, 2, . . . , m. 



Table 4: Problem a): the number of iterations and the relative errors with 
respect to the parameter q (ao = 2, 5 = 1CP 2 ). 



ISx IS 2 





REl.Err 


Iteri 


REl.Err 


Iter; 


.5 


0.008 


12 


0.007 


11 


.25 


0.009 


7 


0.007 


7 


.125 


0.009 


5 


0.008 


5 



Table 5: Problem a): the relative errors and the number of iterations 



8 IS! IS 2 VR 





REl.Err 


Iteri 


REl.Err 


Iter2 


REl.Err 


Iter VR . 


5% 


0.018 


6 


0.014 


6 


0.016 


11 


3% 


0.013 


G 


0.011 


G 


0.013 


12 


1% 


0.009 


7 


0.007 


7 


0.008 


15 



The parameter m = 600 is used in problem a). In this case the condition 
number of the matrix A with to = 600 is 3.427 x 10 9 , so it is an ill-conditioned 
matrix. Here the parameter C in I Si and IS2 are 2 and 1.01, respectively. For 
problem b) the parameter m is 200. In this case the condition number of the 
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Table 6: Problem b): the number of iterations and the relative errors with 
respect to the parameter q (ao = 4, S = 10~ 2 ). 



q 


ISi 




IS 2 






REl.Err 


Iteri 


REl.Err 


Iter 2 ■ 


.5 


0.428 


17 


0.446 


15 


.25 


0.421 


9 


0.436 


9 


.125 


0.439 


6 


0.416 


7 



Table 7: Problem b): the relative errors and the number of iterations 



6 JS*i IS 2 VR 





REl.Err 


Iteri 


REl.Err 


Iter 2 


REl.Err 


ItervR- 


5% 


0.618 


7 


0.621 


7 


0.627 


12 


3% 


0.541 


8 


0.559 


8 


0.584 


13 


1% 


0.421 


9 


0.436 


9 


0.457 


13 



matrix A is 4.863 x 10 4 . The parameter C is 1.01 in the both iterative schemes 
I Si and IS2- In Tables 4 and 6 we give the relation between the parameter q 
and the number of iterations and the relative errors of the iterative schemes 751 
and IS 2 - The closer the parameter q to 1, the larger number of iterations we 
get, and the closer the parameter q to 0, the smaller the number of iterations 
we get. But the relative error starts to increase if the parameter q is chosen 
too small. Based on the numerical results given in Tables 4 and 5, we suggest 
to choose the parameter q in the interval (0.125,0.5). In the iterative schemes 
ISi and IS2 we use the geometric sequence {2 x .2S n ~ 1 }^L 1 for problem a). 
The geometric series {4 x .25 n ~ 1 }^—i is used in problem b). In the variational 
regularization method we use ao = 2 and ao = 4 as the initial regularization 
parameter of the Newton's method in problem a) and 6), respectively. Since 
the Newton's method for solving (|103p is locally convergent, in problem b) we 
need to choose a smaller regularization parameter ao than for ISi and IS2 
methods. Here kg — 8 was used. The numerical results on Table 5 show that 
the solutions produced by the proposed iterative schemes arc stable. In problem 

a) the relative errors of the iterative scheme IS2 , are smaller than these for the 
iterative scheme ISi and than these for the variational regularization, VR. In 
Table 7 the relative errors produced by the three methods for solving problem 

b) are presented. The relative error of ISi is smaller than the one for the other 
two methods. 
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5 Conclusion 

We have demonstrated that the proposed iterative schemes can be used for 
solving ill-conditioned linear algebraic systems stably. The advantage of the 
iterative scheme (fT5j) compared with iterative scheme (f?2"j) is the following: one 
applies the operator T" 1 only once at each iteration. Note that the difficulty of 
using the Newton's method is in choosing the initial value for the regularization 
parameter, since the Newton's method for solving equation (|103p converges 
only locally. In solving (|103[) by the Newton's method one often has to choose 
an initial regularization parameter a$ sufficiently close to the root of equation 
(|103p as shown in problem b) in Section 4.2. In our iterative schemes the initial 
regularization parameter can be chosen in the interval [1,4] which is larger 
than the initial regularization parameter used in the variational regularization 
method. In the iterative scheme ISi we modified the discrepancy-type principle 



given in [IT], by using (TO)) to get discrepancy-type principle (|33|) . which can 
be easily implemented numerically. In Section 3 we used the geometric series 
{aoq n }^ =1 in place of the constant regularization parameter a in the iterative 
scheme 



developed in [5]. This geometric series of the regularization parameter allows 
one to use the a posteriori stopping rule given in ([55]). We proved that this 
stopping rule produces stable approximation of the minimal norm solution of 
equation (TT]) . In all the experiments stopping rules ([3"3"| and ([55]) produce stable 
approximations to the minimal norm solution of equation ([1]) . It is of interest to 
develop a method for choosing the parameter q in the proposed methods which 
gives sufficiently small relative error and small number of iterations. 
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